Network processing resource management in computing systems

ABSTRACT

Embodiments of network processing resource management in computing devices are disclosed therein. In one embodiment, a method includes receiving a request from a network interface controller to perform network processing operations at a first core of a main processor for packets assigned by the network interface controller to a queue of a virtual port of the network interface controller. The method also includes determining whether the first core has a utilization level higher than a threshold when performing the network processing operations to effect processing and transmission of the packets. If the first core has a utilization level higher than the threshold, the method includes issuing a command to the network interface to modify affinitization of the queue from the first core to a second core having a utilization level lower than the threshold.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 15/459,256, filed Mar. 15, 2017, which is anon-provisional application of and claims priority to U.S. ProvisionalApplication No. 62/430,485, filed on Dec. 6, 2016.

BACKGROUND

Remote or “cloud” computing typically utilizes a collection of remoteservers in datacenters to provide computing, data storage, electroniccommunications, or other cloud services. The remote servers can beinterconnected by computer networks to form one or more computingclusters. During operation, multiple remote servers or computingclusters can cooperate to provide a distributed computing environmentthat facilitates execution of user applications to provide cloudservices.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

Servers in datacenters typically include a main processor with multiple“cores” that can operate independently, in parallel, or in othersuitable manners to execute instructions. To facilitate communicationswith one another or with external devices, individual servers can alsoinclude a network interface controller (“NIC”) for interfacing with acomputer network. A NIC typically includes hardware circuitry and/orfirmware configured to enable communications between servers bytransmitting/receiving data (e.g., as packets) via a network mediumaccording to Ethernet, Fibre Channel, Wi-Fi, or other suitable physicaland/or data link layer standards.

During operation, one or more cores of a processor in a server cancooperate with the NIC to facilitate communications to/from softwarecomponents executing on the server. Example software components caninclude virtual machines, applications executing on the virtualmachines, a hypervisor for hosting the virtual machines, or othersuitable types of components. To facilitate communications to/from thesoftware components, the one or more cores can execute suitable networkprocessing operations to enforce communications security, performnetwork virtualization, translate network addresses, maintain acommunication flow state, or perform other suitable functions.

One challenge for improving throughput to the software components on aserver is to overcome limited processing capacities of the cores. Duringoperation, executing network processing operations can overload thecores and thus render the cores as communications bottlenecks. A singlecore is typically used for executing network processing operations for aparticular communication flow in order to maintain a propercommunication flow state such as a proper sequence of transmittedpackets. As available throughput of the NIC increases, a single core canbecome inadequate for executing network processing operations toaccommodate operations of the NIC. As such, processing capabilities ofthe cores can limit transmission rates of data to/from softwarecomponents on the server.

Embodiments of the disclosed technology can address certain aspects ofthe foregoing challenge by implementing multi-stage network processingload balancing in a server having a NIC operatively coupled to multiplecores. In certain embodiments, the NIC can be configured to implement atwo-stage network processing load balancing by having hardwareelectronic circuitry configured to provide (i) a first stage with a portselector configured to select a virtual port; and (ii) a seriallycoupled second stage with a receive side scaling (“RSS”) engineconfigured to further distribute network processing loads. Examples ofsuch hardware electronic circuitry can include an application-specificintegrated circuit (“ASIC”), a field programmable gate array (“FPGA”)with suitable firmware, or other suitable hardware components. A virtualport in a NIC is a virtual network interface corresponding to ahypervisor, a virtual machine, or other components hosted on a server. Avirtual port can include one or more virtual channels (e.g., as queues)individually having an assigned core to accommodate network processingload associated with one or more communication flows (e.g., TCP/UDPflows) such as an exchange of data during a communication sessionbetween two applications on separate servers.

In certain implementations, at the first stage, the port selector can beconfigured to distribute incoming packets to a particular virtual portof the NIC based on a general destination of the incoming packets (e.g.,a virtual machine). In one example, the port selector can be configuredto filter the incoming packets based on a media access control address(“MAC” address) or a combination of a MAC address and a virtual networktag included in headers of the packets. The filtered packets associatedwith a particular MAC address are then assigned to a virtual portassociated with a virtual machine on the server. In otherimplementations, the port selector can be configured to filter theincoming packets based on a virtual machine identifier, a virtualmachine IP address, or other suitable identifiers.

At the second stage, the RSS engine can be configured to furtherdistribute the incoming packets assigned to a virtual port to multiplequeues in the virtual port based on a particular destination of thepackets (e.g., an application executing on the virtual machine). Forexample, in one implementation, the RSS engine can be configured tocalculate a hash value (e.g., 32 bits) based on a source IP address, adestination IP address, a source port, a destination port, and/or othersuitable Transmission Control Protocol (“TCP”) parameters (referred toas “characteristic of communication”) of the packets. The RSS engine canthen assign the packets to a queue in the virtual port based on one ormore bits of the calculated hash value by consulting an indirectiontable associated with the virtual port. The indirection table containsassignments of individual queues with an affinitized or associated corebased on the one or more bits of the hash value. With the identifiedqueue/core, the NIC can then cooperate with the identified core toforward the packets to the particular destination on the server.

Several embodiments of the disclosed technology can improve network datathroughput to applications, virtual machines, or other softwarecomponents on a server when compared to other communication techniques.In certain computing systems, RSS operations can be implemented as asoftware component, for example, a module of an operating systemexecuted by a core on the server. However, using a generic mainprocessor for performing RSS operations such as hash calculations can behighly inefficient. For instance, in one test, a server having softwareimplemented RSS engine could only achieve about 26 Gbit/s of networkdata transmission when the NIC has a capacity of 40 Gbit/s. The softwareimplemented RSS engine can also suffer from performance jitters orvariances when the core experiences operational delays and otherundesirable effects. By offloading execution of RSS operations to thehardware implemented RSS engine in the NIC, data throughput in theserver can be significantly improved. For instance, in another test, aserver having a hardware implemented RSS engine achieved close to 40Gbit/s of network data transmission when the NIC has a capacity of 40Gbit/s.

In other embodiments, a server having a NIC configured to implement thetwo-stage balancing described above can also include a software module(referred to below as “load balancer”) configured to dynamically balancenetwork processing loads on the multiple cores by modifying coreassignments for corresponding queues based on current loads of thecores. For example, the load balancer can be configured to monitor acurrent network processing loads of the cores and compare the currentloads with a high threshold value. In response to determining that acurrent network processing load of a core exceeds the high thresholdvalue, the load balancer can modify one or more entries ofaffinitization or association between a queue and a core in theindirection table of the NIC. The load balancer can also modify one ormore affinitization in the indirection table to combine networkprocessing loads when one or more current loads of the correspondingcores are less than a low threshold. In further examples, the loadbalancer can also be configured to balance the networking processingloads in suitable manners such that a minimal number of cores are usedfor network processing loads. By reducing or limiting the number ofcores used for network processing, power consumption in the server canbe reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a computing system havinghosts implementing network traffic management techniques in accordancewith embodiments of the disclosed technology.

FIG. 2 is a schematic diagram illustrating certain hardware/softwarecomponents of the computing system of FIG. 1 in accordance withembodiments of the disclosed technology.

FIGS. 3A-3F are schematic block diagrams of a host suitable for thecomputing system of FIG. 1 at operational stages during network dataprocessing in accordance with embodiments of the present technology.

FIG. 4 is an example data schema suitable for a header of a packet inaccordance with embodiments of the present technology.

FIG. 5 is a block diagram showing hardware modules suitable for the portselector of FIGS. 3A-3F in accordance with embodiments of the presenttechnology.

FIG. 6 is a block diagram showing hardware modules suitable for the RSSengine of FIGS. 3A-3F in accordance with embodiments of the presenttechnology.

FIG. 7 is a block diagram showing software modules suitable for the loadbalancer of FIGS. 3A-3F in accordance with embodiments of the presenttechnology.

FIGS. 8A-8D are flow diagrams illustrating aspects of processes fornetwork processing resource management in a host in accordance withembodiments of the present technology.

FIG. 9 is a computing device suitable for certain components of thecomputing system in FIG. 1.

DETAILED DESCRIPTION

Various embodiments of computing systems, devices, components, modules,routines, and processes related to network traffic management incomputing devices and systems are described below. In the followingdescription, example software codes, values, and other specific detailsare included to provide a thorough understanding of various embodimentsof the present technology. A person skilled in the relevant art willalso understand that the technology may have additional embodiments. Thetechnology may also be practiced without several of the details of theembodiments described below with reference to FIGS. 1-9.

As used herein, the term a “computing system” generally refers to aninterconnected computer network having a plurality of network devicesthat interconnect a plurality of servers or hosts to one another or toexternal networks (e.g., the Internet). The term “network device”generally refers to a physical network device, examples of which includerouters, switches, hubs, bridges, load balancers, security gateways, orfirewalls. A “host” generally refers to a computing device configured toimplement, for instance, one or more virtual machines or other suitablevirtualized components. For example, a host can include a server havinga hypervisor configured to support one or more virtual machines or othersuitable types of virtual components.

A computer network can be conceptually divided into an overlay networkimplemented over an underlay network. An “overlay network” generallyrefers to an abstracted network implemented over and operating on top ofan underlay network. The underlay network can include multiple physicalnetwork devices interconnected with one another. An overlay network caninclude one or more virtual networks. A “virtual network” generallyrefers to an abstraction of a portion of the underlay network in theoverlay network. A virtual network can include one or more virtual endpoints referred to as “tenant sites” individually used by a user or“tenant” to access the virtual network and associated computing,storage, or other suitable resources. A tenant site can have one or moretenant end points (“TEPs”), for example, virtual machines. The virtualnetworks can interconnect multiple TEPs on different hosts. Virtualnetwork devices in the overlay network can be connected to one anotherby virtual links individually corresponding to one or more networkroutes along one or more physical network devices in the underlaynetwork.

Also used herein, a “packet” generally refers to a formatted unit ofdata carried by a packet-switched or other suitable types of network. Apacket typically includes both control information and user datareferred to as payload. Control information can provide data fortransmitting or delivering a payload. For example, control informationcan include source and destination network addresses, error detectioncodes (e.g., CRC codes), sequencing information, and/or other suitabledata. Typically, control information can be contained in packet headersthat precede the payload and trailers that follow the payload. Anexample header is described below with reference to FIG. 4.

A “virtual port” generally refers to a virtual network interface on aNIC that corresponds to a hypervisor, a virtual machine, or othercomponents hosted on a computing device. A virtual port can include oneor more virtual channels (e.g., as queues) that can be assigned topackets associated with a single communication flow. Each queue can beaffinitized with a single core of a main processor in the server. Theterm “affinitize” generally refers to an assignment, designation, orassociation for establishing a relationship between a queue in a virtualport with a single core in the main processor in the server.

Servers in datacenters typically include a main processor with multiplecores to execute instructions independently, cooperatively, or in othersuitable manners. The servers can also include a NIC for interfacingwith a computer network. The NIC can facilitate, for example,transmission and reception of packets via a network medium according toEthernet, Fibre Channel, Wi-Fi, or other suitable standards. Duringoperation, one or more cores in a server can cooperate with the NIC tofacilitate communications via the computer network. The core can executeinstructions to enforce communications security, perform networkvirtualization, translate network addresses, maintaining a communicationflow state, or perform other suitable functions.

One challenge for improving throughput to virtual machines orapplications executing in the virtual machines on a server is that thecores can be overloaded with executing the network processing operationsor loads and become communications bottlenecks. Typically, a single coreis used for executing network processing loads for a communication flowto maintain a proper communication flow state, e.g., a proper sequenceof transmitted packets. As available throughput of the NIC increases, asingle core can have inadequate processing capability to execute thenetwork processing loads to accommodate the throughput of the NIC. Assuch, processing capabilities of the cores can limit transmission ratesof network data to/from applications, virtual machines, or othersoftware components executing on the servers.

Several embodiments of the disclosed technology can address certainaspects of the foregoing challenge by implementing multi-stage networkprocessing load balancing in a server having a NIC operatively coupledto multiple cores of a processor in a server. In certain embodiments,the NIC can be configured to implement two-stage hardware networkprocessing load balancing by having (i) a first stage with a portselector and, in series with the first stage, (ii) a second stage with areceive side scaling (“RSS”) engine. At the first stage, the portselector can be configured to distribute incoming packets to aparticular virtual port of the NIC based on MAC addresses of theincoming packets. At the second stage, the RSS engine can be configuredto further distribute the incoming packets assigned to a virtual port tomultiple queues in the virtual port based on characteristic ofcommunication of the packets. With the identified queue/core, the NICcan then cooperate with the identified core to forward the packets tosuitable applications, virtual machines, or other software components onthe server.

The network processing loads can be further distributed or coalesced byutilizing a software implemented load balancer. The load balancer can beconfigured to dynamically balance network processing loads on themultiple cores by modifying core affinitization of queues based oncurrent network processing loads of the cores. For example, the loadbalancer can be configured to monitor a current network processing loadsof the cores and compare the current loads with a high threshold value.In response to determining that a current network processing load of acore exceeds the high threshold value, the load balancer can “relocate”network processing load of a particular queue to a new core. In otherexamples, the load balancer can also be configured to combine networkprocessing loads when one or more current loads of the correspondingcores are less than a low threshold. As such, the load balancer canbalance networking processing loads on a number of cores such that aminimal number of cores are used for network processing loads. Byreducing or limiting the number of cores used for network processing,power consumption in the server can be reduced, as described in moredetail below with reference to FIGS. 1-9.

FIG. 1 is a schematic diagram illustrating a computing system 100 havinghosts implementing network traffic management techniques in accordancewith embodiments of the disclosed technology. As shown in FIG. 1, thecomputing system 100 can include an underlay network 108 interconnectinga plurality of hosts 106, a plurality of client devices 102 of tenants101 to one another. Even though particular components of the computingsystem 100 are shown in FIG. 1, in other embodiments, the computingsystem 100 can also include network storage devices, maintenancemanagers, and/or other suitable components (not shown) in addition to orin lieu of the components shown in FIG. 1.

As shown in FIG. 1, the underlay network 108 can include multiplenetwork devices 112 that interconnect the multiple hosts 106 and theclient devices 102. In certain embodiments, the hosts 106 can beorganized into racks, action zones, groups, sets, or other suitabledivisions. For example, in the illustrated embodiment, the hosts 106 aregrouped into three host sets identified individually as first, second,and third host sets 107 a-107 c. In the illustrated embodiment, each ofthe host sets 107 a-107 c is operatively coupled to correspondingnetwork devices 112 a-112 c, respectively, which are commonly referredto as “top-of-rack” or “TOR” network devices. The TOR network devices112 a-112 c can then be operatively coupled to additional networkdevices 112 to form a computer network in a hierarchical, flat, mesh, orother suitable types of topology. The computer network can allowcommunications among the hosts 106 and the client devices 102. In otherembodiments, the multiple host sets 107 a-107 c can share a singlenetwork device 112 or can have other suitable arrangements.

The hosts 106 can individually be configured to provide computing,storage, and/or other suitable cloud computing services to theindividual tenants 101. For example, as described in more detail belowwith reference to FIG. 2, each of the hosts 106 can initiate andmaintain one or more virtual machines 144 (shown in FIG. 2) uponrequests from the tenants 101. The tenants 101 can then utilize theinstantiated virtual machines 144 to perform computation, communication,and/or other suitable tasks. In certain embodiments, one of the hosts106 can provide virtual machines 144 for multiple tenants 101. Forexample, the host 106 a can host three virtual machines 144 individuallycorresponding to each of the tenants 101 a-101 c. In other embodiments,multiple hosts 106 can host virtual machines 144 for the tenants 101a-101 c.

The client devices 102 can each include a computing device thatfacilitates corresponding users 101 to access cloud services provided bythe hosts 106 via the underlay network 108. For example, in theillustrated embodiment, the client devices 102 individually include adesktop computer. In other embodiments, the client devices 102 can alsoinclude laptop computers, tablet computers, smartphones, or othersuitable computing devices. Even though three users 101 are shown inFIG. 1 for illustration purposes, in other embodiments, the distributedcomputing system 100 can facilitate any suitable number of users 101 toaccess cloud or other suitable types of computing services provided bythe hosts 106.

FIG. 2 is a schematic diagram illustrating an overlay network 108′implemented on the underlay network 108 in FIG. 1 in accordance withembodiments of the disclosed technology. In FIG. 2, only certaincomponents of the underlay network 108 of FIG. 1 are shown for clarity.As shown in FIG. 2, the first host 106 a and the second host 106 b caneach include a processor 132, a memory 134, and a network interface 136operatively coupled to one another. The processor 132 can include one ormore microprocessors, field-programmable gate arrays, and/or othersuitable logic devices. The memory 134 can include volatile and/ornonvolatile media (e.g., ROM; RAM, magnetic disk storage media; opticalstorage media; flash memory devices, and/or other suitable storagemedia) and/or other types of computer-readable storage media configuredto store data received from, as well as instructions for, the processor132 (e.g., instructions for performing the methods discussed below withreference to FIG. 8A-8D). The network interface 136 can include a NIC, aconnection converter, and/or other suitable types of input/outputdevices configured to accept input from and provide output to othercomponents on the virtual networks 146.

The first host 106 a and the second host 106 b can individually containinstructions in the memory 134 executable by the processors 132 to causethe individual processors 132 to provide a hypervisor 140 (identifiedindividually as first and second hypervisors 140 a and 140 b). Thehypervisors 140 can be individually configured to generate, monitor,terminate, and/or otherwise manage one or more virtual machines 144organized into tenant sites 142. For example, as shown in FIG. 2, thefirst host 106 a can provide a first hypervisor 140 a that manages firstand second tenant sites 142 a and 142 b, respectively. The second host106 b can provide a second hypervisor 140 b that manages first andsecond tenant sites 142 a′ and 142 b′, respectively. The hypervisors 140are individually shown in FIG. 2 as a software component. However, inother embodiments, the hypervisors 140 can also include firmware and/orhardware components. The tenant sites 142 can each include multiplevirtual machines 144 for a particular tenant 101 (FIG. 1). For example,the first host 106 a and the second host 106 b can both host the tenantsite 142 a and 142 a′ for a first tenant 101 a (FIG. 1). The first host106 a and the second host 106 b can both host the tenant site 142 b and142 b′ for a second tenant 101 b (FIG. 1). Each virtual machine 144 canbe executing a corresponding operating system, middleware, and/orsuitable applications. The executed applications can each correspond toone or more cloud computing services or other suitable types ofcomputing services.

Also shown in FIG. 2, the computing system 100 can include an overlaynetwork 108′ having one or more virtual networks 146 that interconnectthe tenant sites 142 a and 142 b across the first and second hosts 106 aand 106 b. For example, a first virtual network 142 a interconnects thefirst tenant sites 142 a and 142 a′ at the first host 106 a and thesecond host 106 b. A second virtual network 146 b interconnects thesecond tenant sites 142 b and 142 b′ at the first host 106 a and thesecond host 106 b. Even though a single virtual network 146 is shown ascorresponding to one tenant site 142, in other embodiments, multiplevirtual networks (not shown) may be configured to correspond to a singletenant site 146.

The virtual machines 144 on the virtual networks 146 can communicatewith one another via the underlay network 108 (FIG. 1) even though thevirtual machines 144 are located or hosted on different hosts 106.Communications of each of the virtual networks 146 can be isolated fromother virtual networks 146. In certain embodiments, communications canbe allowed to cross from one virtual network 146 to another through asecurity gateway or otherwise in a controlled fashion. A virtual networkaddress can correspond to one of the virtual machine 144 in a particularvirtual network 146. Thus, different virtual networks 146 can use one ormore virtual network addresses that are the same. Example virtualnetwork addresses can include IP addresses, MAC addresses, and/or othersuitable addresses.

In operation, the hosts 106 can facilitate communications among thevirtual machines and/or applications executing in the virtual machines144. For example, the processor 132 can execute suitable networkcommunication operations to facilitate the first virtual machine 144′ totransmit packets to the second virtual machine 144″ via the virtualnetwork 146 a by traversing the network interface 136 on the first host106 a, the underlay network 108 (FIG. 1), and the network interface 136on the second host 106 b. In accordance with embodiments of thedisclosed technology, the network interfaces 136 can be implemented withmulti-stage network processing load balancing to improve throughput tothe virtual machines 144 and/or applications (not shown) executing inthe virtual machines 144, as described in more detail below withreference to FIGS. 3A-3F.

FIGS. 3A-3F are schematic block diagrams of a host 106 suitable for thecomputing system 100 of FIG. 1 at various operational stages duringnetwork data processing in accordance with embodiments of the presenttechnology. In particular, FIGS. 3A-3C illustrate operational stagesrelated to distribution of network processing loads to additionalcore(s) for multiple communication flows. As used herein, a“communication flow” generally refers to a sequence of packets from asource (e.g., an application or a virtual machine executing on a host)to a destination, which can be another application or virtual machineexecuting on another host, a multicast group, or a broadcast domain.FIGS. 3C-3F illustrate operational stages related to coalescing networkprocessing loads to one or more cores for multiple for multiplecommunication flows. Though particular components of the host 106 aredescribed below, in other embodiments, the host 106 can also includeadditional and/or different components in lieu of or in additional tothose shown in FIGS. 3A-3F. Details of the various operational stagesare described below in turn.

In FIGS. 3A-3F and in other Figures herein, individual softwarecomponents, objects, classes, modules, and routines may be a computerprogram, procedure, or process written as source code in C, C++, C#,Java, and/or other suitable programming languages. A component mayinclude, without limitation, one or more modules, objects, classes,routines, properties, processes, threads, executables, libraries, orother components. Components may be in source or binary form. Componentsmay also include aspects of source code before compilation (e.g.,classes, properties, procedures, routines), compiled binary units (e.g.,libraries, executables), or artifacts instantiated and used at runtime(e.g., objects, processes, threads).

Components within a system may take different forms within the system.As one example, a system comprising a first component, a secondcomponent, and a third component. The foregoing components can, withoutlimitation, encompass a system that has the first component being aproperty in source code, the second component being a binary compiledlibrary, and the third component being a thread created at runtime. Thecomputer program, procedure, or process may be compiled into object,intermediate, or machine code and presented for execution by one or moreprocessors of a personal computer, a tablet computer, a network server,a laptop computer, a smartphone, and/or other suitable computingdevices.

Equally, components may include hardware circuitry. In certain examples,hardware may be considered fossilized software, and software may beconsidered liquefied hardware. As just one example, softwareinstructions in a component may be burned to a Programmable Logic Arraycircuit, or may be designed as a hardware component with appropriateintegrated circuits. Equally, hardware may be emulated by software.Various implementations of source, intermediate, and/or object code andassociated data may be stored in a computer memory that includesread-only memory, random-access memory, magnetic disk storage media,optical storage media, flash memory devices, and/or other suitablecomputer readable storage media. As used herein, the term “computerreadable storage media” excludes propagated signals.

As shown in FIG. 3A, the host 106 can include a motherboard 111 carryinga processor 132, a main memory 134, and a network interface 135operatively coupled to one another. Though not shown in FIGS. 3A-3F, inother embodiments, the host 106 can also include a memory controller, apersistent storage, an auxiliary power source, a baseboard managementcontroller operatively coupled to one another. In certain embodiments,the motherboard 111 can include a printed circuit board with one or moresockets configured to receive the foregoing or other suitable componentsdescribed herein. In other embodiments, the motherboard 111 can alsocarry indicators (e.g., light emitting diodes), platform controllerhubs, complex programmable logic devices, and/or other suitablemechanical and/or electric components in lieu of or in addition to thecomponents shown in FIGS. 3A-3F.

The processor 132 can be an electronic package containing variouscomponents configured to perform arithmetic, logical, control, and/orinput/output operations. The processor 132 can be configured to executeinstructions to provide suitable computing services, for example, inresponse to a user request received from the client device 102 (FIG. 1).As shown in FIG. 3A, the processor 132 can include one or more “cores”133 configured to execute instructions independently or in othersuitable manners. Four cores 133 (illustrated individually as first,second, third, and fourth cores 133 a-133 d, respectively) are shown inFIG. 3A for illustration purposes. In other embodiments, the processor132 can include eight, sixteen, or any other suitable number of cores133. The cores 133 can individually include one or more arithmetic logicunits, floating-point units, L1 and L2 cache, and/or other suitablecomponents. Though not shown in FIG. 3A, the processor 132 can alsoinclude one or more peripheral components configured to facilitateoperations of the cores 133. The peripheral components can include, forexample, QuickPath® Interconnect controllers, L3 cache, snoop agentpipeline, and/or other suitable elements.

The main memory 134 can include a digital storage circuit directlyaccessible by the processor 132 via, for example, a data bus 131. In oneembodiment, the data bus 131 can include an inter-integrated circuit busor I²C bus as detailed by NXP Semiconductors N.V. of Eindhoven, theNetherlands. In other embodiments, the data bus 131 can also include aPCIe bus, system management bus, RS-232, small computer system interfacebus, or other suitable types of control and/or communications bus. Incertain embodiments, the main memory 134 can include one or more DRAMmodules. In other embodiments, the main memory 134 can also includemagnetic core memory or other suitable types of memory.

As shown in FIG. 3A, the processor 132 can cooperate with the mainmemory 134 to execute suitable instructions to provide one or morevirtual machines 144. In FIG. 3A, two virtual machines 144 (illustratedas first and second virtual machines 144 a and 144 b, respectively) areshown for illustration purposes. In other embodiments, the host 106 canbe configured to provide one, three, four, or any other suitable numberof virtual machines 144. The individual virtual machines 144 can beaccessible to the tenants 101 (FIG. 1) via the overlay and underlaynetwork 108′ and 108 (FIGS. 1 and 2) for executing suitable useroperations. For example, as shown in FIG. 3A, the first virtual machine144 a can be configured to execute applications 147 (illustrated asfirst and second applications 147 a and 147 b, respectively) for one ormore of the tenants 101 in FIG. 1. In other examples, the individualvirtual machines 144 can be configured to execute multiple applications147, as described in more detail below with respect to FIGS. 3D-3F.

The individual virtual machines 144 can include a corresponding virtualinterface 145 (identified as first virtual interface 145 a and secondvirtual interface 145 b) for receiving/transmitting data packets via thevirtual network 108′. In certain embodiments, the virtual interfaces 145can each be a virtualized representation of resources at the networkinterface 136 (or portions thereof). For example, the virtual interfaces145 can each include a virtual Ethernet or other suitable types ofinterface that shares physical resources at the network interface 136.Even though only one virtual interface 145 is shown for each virtualmachine 144, in further embodiments, a single virtual machine 144 caninclude multiple virtual interfaces 145 (not shown).

As shown in FIG. 3A, the processor 132 can cooperate with the mainmemory 134 to execute suitable instructions to provide a load balancer130. In the illustrated embodiment, the first core 133 a is shown asexecuting and providing the load balancer 130. In other embodiments,other suitable core(s) 133 can also be tasked with executing suitableinstructions to provide the load balancer 130. In certain embodiments,the load balancer 130 can be configured to monitor status of networkprocessing loads on the cores 133 and dynamically re-affinitize orre-assign cores for executing network processing loads for particularqueues 139.

In one embodiment, the load balancer 130 can be configured to distributenetwork processing loads currently carried by a particular core 133 tomultiple cores 133. For example, the load balancer 133 can receive andcompare a current utilization value (e.g., a percentage or fraction) ofa core 133 with a high threshold (e.g., 90% or 95%). If the currentutilization value of the core 133 exceeds the high thresholdinstantaneously or over a preset period, the load balancer 130 can beconfigured to determine (i) which queue(s) 139 (or associatedcommunication flows) can be relocated from the current core 133; and(ii) whether another new core 133 has capacity to assume responsibilityfor executing network processing loads associated with the queue(s) 139.In some implementations, the new core 133 can be selected based onprocessor cache-proximity to either the current core 133, or a“preferred” core 133 that is selected based on performanceconsiderations. For example, the preferred core 133 can be a core 133 onwhich a VM virtual processor that handles the queue 139 is running.Thus, in certain examples, the new core 133 can be selected from cores133 residing on the same L1 cache with either the preferred or currentcore 133. If no acceptable core 133 on the same L1 cache is acceptable,the new core 133 can be selected from cores 133 residing on the same L2cache as the preferred or previous core 133. If still no acceptable core133 is found, the new core 133 can be selected from cores 133 sharing L3cache. If still no acceptable core 133 is found, all cores 133 onpreferred non-uniform memory access (“NUMA”) may be considered as thenew core 133. Upon determination, the load balancer 130 can thenre-affinitize the queue(s) 139 with one or more additional cores 133 inthe network interface 136. As such, network processing loads of thecores 133 can be distributed to prevent or at least reduce the risk ofthe particular core 133 becoming a communication bottleneck.

In another embodiment, the load balancer 130 can also be configured tocoalesce network processing loads of multiple queues 139 on a particularcore 133. Thus, fewer number of cores 133 can be operating than beforesuch coalescence. In one implementation, the load balancer 130 can beconfigured to compare the current utilization value of a core 133 with alow threshold (e.g., 15%, 20%, or other suitable values). If the currentutilization value of the core 133 is lowered than the low threshold, theload balancer 130 can be configured to determine if another core 133 hascapacity to assume responsibility for executing network processing loadscarried by the core 133 without exceeding the high threshold (or othersuitable thresholds). Upon determination, the load balancer 130 can beconfigured to re-affinitize any queue(s) 139 associated with the core133 with the another core 133 in the network interface 136. As such, thecore 133 can be shut down, enter a power save mode, or otherwise reducepower consumption. Example of operations of the load balancer 130 aredescribed in more detail below with reference to FIGS. 3B and 3C.

The network interface 136 can be configured to facilitate virtualmachines 144 and/or applications 147 executing on the host 106 tocommunicate with other components (e.g., other virtual machines 144 onother hosts 106) on the virtual networks 146 (FIG. 2). In FIGS. 3A-3F,hardware components are illustrated with solid lines while softwarecomponents are illustrated in dashed lines. In certain embodiments, thenetwork interface 136 can include one or more NICs. One suitable NIC forthe network interface 136 can be a HP InfiniBand FDR/EN 10/40 Gb DualPort 544FLR-QSFP Network Adapter provided by Hewlett-Packard of PaloAlto, Calif. In other embodiments, the network interface 136 can alsoinclude port adapters, connectors, or other suitable types of networkcomponents in addition to or in lieu of a NIC. Though only one NIC isshown in FIG. 3A as an example of the network interface 136, in furtherembodiments, the host 106 can include multiple NICs (not shown) of thesame or different configurations to be operated in parallel or in othersuitable manners.

As shown in FIG. 3A, the network interface 136 can include a controller122, a memory 124, and one or more virtual ports 138 operatively coupledto one another. The controller 122 can include hardware electroniccircuitry configured to receive and transmit data,serialize/de-serialize data, and/or perform other suitable functions tofacilitate interfacing with other devices on the virtual networks 146.Suitable hardware electronic circuitry suitable for the controller 122can include a microprocessor, an ASIC, a FPGA, or other suitablehardware components. Example modules for the controller 122 aredescribed in more detail below. The memory 124 can include volatileand/or nonvolatile media (e.g., ROM; RAM, flash memory devices, and/orother suitable storage media) and/or other types of computer-readablestorage media configured to store data received from, as well astransmitted to other components on the virtual networks 146.

The virtual ports 138 can be configured to interface with one or moresoftware components executing on the host 106. For example, as shown inFIG. 3A, the network interface 136 can include two virtual ports 138(identified as first and second virtual ports 138 a and 138 b,respectively) individually configured to interface with the first andsecond virtual machines 144 a and 144 b via the first and second virtualinterfaces 145 a and 145 b, respectively. As such, communication flowsto the first virtual machine 144 a pass through the first virtual port138 a while communication flows to the second virtual machine 144 b passthrough the second virtual port 138 b.

As shown in FIG. 3A, each of the virtual ports 138 can include multiplechannels or queues 139 individually configured to handle one or morecommunication flows. In the illustrated embodiment in FIG. 3A, the firstvirtual port 138 a includes three queues 139 (identified individually asfirst, second, and third queues 139 a-139 c, respectively). The secondvirtual port 138 b includes two queues 139 (identified individually asfirst and second queues 139 a′ and 139 b′, respectively). In otherembodiments, the first and/or second virtual ports 138 can include four,five, six, or any other suitable number of queues 139.

The individual queues 139 can be affinitized or associated with (asindicated by the arrows 135) one of the cores 133 for executing networkprocessing operations for a communication flow through a correspondingqueue 139. For example, in the illustrated embodiment, the first,second, and third queues 139 a-139 c in the first virtual port 138 a areaffinitized to the second core 133 b. The first and second queues 139 a′and 139 b′ of the second virtual port 138 b are affinitized with thethird and fourth cores 133 c and 133 d, respectively. In otherembodiments, the foregoing queues 139 in the virtual ports 138 can beaffinitized with other cores 133 in any suitable manners. In furtherembodiments, the foregoing affinitization or association between theindividual queues 139 and the cores 133 can be dynamically adjusted by,for example, by the load balancer 130, as described in more detaillater.

As shown in FIG. 3A, the controller 122 can include a media access unit(“MAU”) 123, a packet handler 125, a port selector 126, an affinityagent 127, and a RSS engine 128 operatively coupled to one another.Though particular components are shown in FIG. 3A, in other embodiments,the controller 122 can also include direct memory access interfaceand/or other suitable components. The MAU 123 can be configured tointerface with a transmission medium of the underlay network 108(FIG. 1) to receive and/or transmit data, for example, as packets 150having a header, a payload, and optionally a trailer. In one embodiment,the MAU 123 can include an Ethernet transceiver. In other embodiments,the MAU 123 can also include a fiber optic transceiver or other suitabletypes of media interfacing components.

The packet handler 125 can be configured to facilitate operationsrelated to receiving and transmission of packets 150. For example, incertain embodiments, the packet handler 125 can include a receivede-serializer, a CRC generator/checker, a transmit serializer, anaddress recognition module, a first-in-first-out control module, and aprotocol control module. In other embodiments, the packet handler 125can also include other suitable modules in addition to or in lieu of theforegoing modules. As described in more detail below, the packet handler125 can also cooperate with the port selector 126 and the RSS engine 128to process and forward packets 150 to the virtual machines 144 and/orthe application 147.

The affinity agent 127 can be configured to modify affinitizationbetween the queues 139 and the cores 133 on the network interface 136.The affinity agent 127 can be configured to provide to the processor132, or an operating system (not shown) executing on the processor 132 adefault affinitization between the queues 139 and the cores 133. Theaffinity agent 127 can also be configured to indicate to the processor132 or the operating system that the default affinitization can bemodified via, for example, an application programming interface (“API”)or other suitable types of hardware/software interface. In response tosuitable instructions, the affinity agent 127 can be configured tomodify, reset, or otherwise adjust affinitization between the queues 139and the cores 133. Certain examples of such modification are describedbelow with reference to FIGS. 3B-3F.

In accordance with embodiments of the disclosed technology, the networkinterface 136 can be implemented with two-stage network processing loadbalance by utilizing the port selector 126 as a first stage and the RSSengine 128 as a second stage implemented in the hardware electroniccircuitry of the controller 122. The port selector 126 can be configuredto distribute incoming packets 150 to a particular virtual port 138 ofthe network interface 136 by identifying a general destination of theincoming packets 150 (e.g., a virtual machine 144). For example, theport selector 126 can be configured to filter the incoming packets 150based on a media access control address (“MAC” address) included inheaders of the packets 150. The filtered packets 150 associated with aparticular MAC address are then assigned to a virtual port 138associated with a virtual machine 144 on the host 106. For instance, asshown in FIG. 3A, the port selector 126 can identify that the incomingpackets 150 and 150′ are destined to the first virtual machine 144 abased on a MAC address contained in headers of the packets 150 and 150′.In response, the port selector 126 can assign the packets 150 and 150′temporarily held in the memory 124 to the first virtual port 138 a, asindicated by the arrow 137. In other implementations, the port selector126 can be configured to filter the incoming packets 150 and 150′ basedon a virtual machine identifier, a virtual machine IP address, or othersuitable identifiers. Example implementations of the port selector 126are described below with reference to FIG. 5.

As shown in FIG. 3B, the RSS engine 128 can be configured to furtherdistribute the incoming packets 150 and 150′ assigned to a virtual port138 to a particular queue 139 in the virtual port 138 based on aparticular destination of the packets 150 and 150′ (e.g., theapplication 147 executing on the virtual machine 144). For example, theRSS engine 128 can be configured to calculate a hash value (e.g., 32bits) based on a source IP address, a destination IP address, a sourceport, a destination port, and/or other suitable Transmission ControlProtocol (“TCP”) parameters (referred to as “characteristic ofcommunication”) included in the headers of the packets 150 and 150′. Anexample header structure suitable for the packets 150 and 150′ isdescribed below with reference to FIG. 4.

Upon identifying the particular destination, the RSS engine 128 can thenassign the packets 150 and 150′ to one or more queues 139 in the virtualport 138 based on one or more bits of the calculated hash value byconsulting an indirection table associated with the virtual port 138.The indirection table can be contained in the memory 124, a persistentstorage (not shown), or in other suitable locations of the networkinterface 136. The indirection table can contain assignments orotherwise indicate the affinitized cores 133 with the individual queues139 based on the one or more bits of the hash value. The following is anexample indirection table for the illustrated example of the firstvirtual port 138 a in FIG. 3B using two bits from the calculated hashvalue:

Bit value Queue Number Core Number 00 1 2 01 2 2 10 3 2

In the illustrated example, the RSS engine 128 selects the second queue139 b (shown in reverse contrast) for the packets 150 and selects thethird queue 139 c for the packets 150′ based on the characteristic ofcommunication of the packets 150 and 150′. In other examples, the RSSengine 128 can select another suitable queue 139 in the first virtualport 138 a. As shown in FIG. 3B, both the identified second and thirdqueues 139 b and 139 c are affinitized with the second core 133 b (alsoshown in reverse contrast). As such, the second core 133 b is taskedwith executing network processing loads for both the packets 150 and150′ in the second and third queues 139 b and 139 c. Exampleimplementations of the RSS engine 128 are described below with referenceto FIG. 6.

With the identified queue/core 139/133, the packet handler 125 of thenetwork interface 136 can then cooperate with the identified second core133 b to forward the packets 150 and 150′ to the particular destinationon the host 106. In certain implementations, the packet handler 125 candetect that a certain amount of data (e.g., a number of packets 150 and150′) have been received in the second and third queues 139 b and 139 c,respectively. In response, the packet handler 125 can generate aninterrupt to the processor 132 (and/or an operation system executing bythe processor 132) to schedule a remote procedure call on the secondcore 133 b. Once the scheduled remote procedure call executes on thesecond core 133 b, the second core 133 b can inspect and retrieve anypackets 150 and 150′ from the second and third queues 139 b and 139 c,perform suitable processing on the retrieved packets 150 and 150′, andforward the processed packets 150 and 150′ to the virtual machine 144associated with the virtual port 138, e.g., the first virtual machine144 a in FIG. 3B. The first virtual machine 144 a can then forward thereceived packets 150 and 150′ to the first and second applications 147 aand 147 b, respectively, for further processing. In otherimplementations, the packet handler 125 can initiate the networkprocessing operations by the second core 133 b in other suitablemanners.

In operation, the MAU 123 receives the packets 150 and 150′ via theunderlay network 108 (FIG. 1) and temporarily stores a copy of thereceived packets 150 and 150′ in the memory 124 in cooperation with thepacket handler 125. The port selector 126 can then inspect a portion ofthe headers of the packets 150 and 150′ for a general destination of thepackets 150 and 150′ by identifying, for example, a MAC addresscorresponding to one of the virtual machines 144. Based on theidentified general destination, the port selector 126 can assign thepackets 150 and 150′ to a virtual port 138, e.g., the first virtual port138 a. Once assigned to a virtual port 138, the RSS engine 128 can thenselect one of the queues 139 in the virtual port 138 for handling thepackets 150 and 150′ based on, for example, a communicationcharacteristic of the packets 150 and 150′. Upon detecting that acertain number of packets 150 and 150′ are in the assigned queue 139,the packet handler 125 can then generate and transmit an interrupt tothe processor 132 to schedule and/or initiate the network processingoperations associated with the packets 150 and 150′.

During operation, the second core 133 b can be overloaded with executionof network processing loads for processing the packets 150 and 150′ fromboth the second and third queues 139 b and 139 c. For example, as shownin FIG. 3B, the second core 133 b can have a utilization percentage 149that exceeds a high threshold (e.g., 90% or 95%). Under such operatingconditions, the second core 133 b can become a communication bottleneckfor processing packets 150 and 150′ in the second and third queues 139 band 139 c.

In accordance with embodiments of the disclosed technology, the loadbalancer 130 can monitor for such conditions and further distributenetwork processing loads to additional cores 133. In certainembodiments, the load balancer 130 can monitor utilization percentageand/or other operating parameters of the individual cores 133, forexample, via a debug port on the uncore or other suitable interfaces ofthe processor 132. In other embodiments, the load balancer 130 canreceive a notification from the processor 132. The notification canindicate to the load balancer 130 that a utilization percentage of thesecond core 133 b exceeds a threshold (e.g., 75%) and a current value ofthe utilization percentage. In further embodiments, the load balancer130 can monitor operating parameters of the cores 133 in other suitablemanners.

Based on the received information, the load balancer 130 can calculatean overall utilization for each core 133, a total time spent inexecuting network processing loads for each queue 139, a total number ofpackets processed for each queue 139, and/or other suitable operatingvalues. Using such received and/or calculated operatingparameters/values, the load balancer 130 can determine whether any ofthe cores 133 is overloaded and thus susceptible to become acommunication bottleneck. As such, in the example illustrated in FIG.3B, the load balancer 130 can determine that the second core 133 b isoverloaded by having a utilization percentage exceeding a highthreshold.

Upon such a determination, the load balancer 130 can then determine (i)which queue(s) 139 (or associated communication flows) can be relocatedfrom the second core 133 b; and (ii) if another core 133 has capacity toassume responsibility for executing network processing loads associatedwith the second and third queues 139 b and 139 c. In certainembodiments, the load balancer 130 can select a queue 139 with thelowest or highest network processing loads to be relocated. In otherembodiments, the load balancer 130 can select a queue 139 to relocatebased on other suitable criteria. The load balancer 130 can also selectanother core 133 as a destination for relocating the queue 139 based onvarious conditions. For example, the load balancer 130 can select acurrently idle core 133 (e.g., with a utilization percentage lower thana preset threshold) as the destination. In other examples, the loadbalancer 130 can select another core 133 by default or based on othersuitable conditions.

Once selected, the load balancer 130 can transmit a modification command190 to the network interface 136 for modifying affinitization between aqueue 139 and a core 133. For example, as shown in FIG. 3B, themodification command 190 can instruct the affinity agent 127 to changeaffinitization of the third queue 139 c of the first virtual port 138 afrom the second core 133 b to the third core 133 c. In response, theaffinity agent 127 can modify the indirection table accordingly. Thus,the example indirection table related to the first virtual port 138 adiscussed above would be modified as follows:

Bit value Queue Number Core Number 00 1 2 01 2 2 10 3 3

As such, the third queue 139 c is now affinitized with the third core133 c to execute network processing loads for the third queue 139 c. Asshown in FIG. 3C, by relocating the network processing loads of thethird queue 139 c from the second core 133 b to the third core 133 c,the utilization percentages 149 and 149′ of both the second and thirdcores 133 b and 133 c can be lower than the high threshold, and thuspreventing these cores 133 to become communication bottlenecks.

Even though one communication flow is relocated in FIGS. 3A-3C toillustrate redistribution of execution of network processing loads, inother embodiments, the load balancer 130 can also redistributeadditional communication flows from the second core 133 b based on atarget utilization percentage (or other suitable operation parameters).For example, the load balancer 130 can iterative relocate communicationflows from the second core 133 b until a utilization percentage of thesecond core 133 b is below a target threshold. In other examples, theload balancer 130 can perform such redistribution in other suitablemanners.

FIGS. 3C-3F illustrate operational stages related to coalescing networkprocessing loads to one or a reduced number of cores 133 for multiplecommunication flows. As shown in FIG. 3C, the port selector 126 candistribute the packets 150 and 150′ to different virtual ports 138 basedon, for instance, MAC addresses included in the headers of the packets150 and 150′. In the illustrated example, the packets 150 is assigned tothe first virtual port 138 a while the packets 150′ are assigned to thesecond virtual port 138 b. As shown in FIG. 3E, the RSS engine 128 canthe select one of the queues 139 for receiving the packets 150 and 150′.In the illustrated example, the third queue 139 b of the first virtualport 138 a is selected to receive the packets 150. The first queue 139a′ of the second virtual port 138 b is selected to receive the packets150′.

As shown in FIG. 3C, the third queue 139 b of the first virtual port 138a and the first queue 139 a′ of the second virtual port 138 b areaffinitized with the second and third cores 133 b and 133 c,respectively. As such, the second and third cores 133 b and 133 c canexecute network processing loads to facilitate processing of the packets150 and 150′ from the third queue 139 b of the first virtual port 138 aand the first queue 139 a′ of the second virtual port 138 b,respectively. During operation, the load balancer 130 can determine thata utilization percentage 149′ of the second core 133 b falls below a lowthreshold (e.g., 15%) when executing network processing loads for thepackets 150. In response, the load balancer 130 can determine that thethird core 133 c can accommodate network processing loads of the packets150 without exceeding a utilization threshold (e.g., the high thresholddiscussed above or other suitable thresholds).

The load balancer 130 can then issue a modification command 190instructing the affinity agent 127 to modify the indirection table suchthat the second queue 139 b of the first virtual port 138 a isaffinitized with the third core 133 c. As shown in FIG. 3F, the thirdcore 133 c can then execute network processing loads for processing thepackets 150 and 150′ from both the second queue 139 b of the firstvirtual port 138 a and the first queue 139 a′ from the second virtualport 138 b. Thus, the second core 133 b can be shut down, enter a powersave or other suitable modes to reduce power consumption.

Even though two communication flows are used in FIGS. 3C-3F toillustrate coalescence of execution of network processing loads, inother embodiments, the load balancer 130 can also coalesce additionalcommunication flows from other queues 139 such that a minimum or targetnumber of cores 133 are used to execute suitable network processingloads for the packets in any queues 139. The load balancer 130 canperform such coalescence in series, in batch, or in other suitablemanners.

Several embodiments of the disclosed technology can improve network datathroughput to applications 147, virtual machines 144, or other softwarecomponents on a host 106 when compared to other communicationtechniques. In certain computing systems, RSS operations can beimplemented as a software component, for example, a module of anoperating system executed by a core on the server. However, using ageneric main processor for performing RSS operations such as hashcalculations can be highly inefficient. For instance, in one test, aserver having software implemented RSS engine could only achieve about26 Gbit/s of network data transmission when the NIC has a capacity of 40Gbit/s. The software implemented RSS engine can also suffer fromperformance jitters or variances when the core experiences operationaldelays and other undesirable effects. By offloading execution of RSSoperations to the hardware implemented RSS engine 128 in the networkinterface 136, data throughput in the host 106 can be significantlyimproved. For instance, in another test, a server having a hardwareimplemented RSS engine 128 achieved close to 40 Gbit/s of network datatransmission when the NIC has a capacity of 40 Gbit/s.

FIG. 4 is an example data schema suitable for a header 160 of a packetin accordance with embodiments of the present technology. In addition tothe header 160, the packet can also include a payload and a trailer (notshown). As shown in FIG. 4, the header 160 can include MAC addresses 162(i.e., destination MAC 162 a and source MAC 162 b), an IP header 164(i.e., destination IP address 164 a and source IP address 164 b), and aTCP header 166 (i.e., destination port 166 a and source port 166 b). Incertain embodiments, the combination of the IP header 164 and the TCPheader 166 is referred to as a characteristic of communication 168 of apacket associated with the header 160. In other embodiments, otherheader fields (not shown) can also be a part of the characteristic ofcommunication 168 in addition to or in lieu of the IP header 164 and theTCP header 166.

FIG. 5 is a block diagram showing example hardware modules suitable forthe port selector 126 of FIGS. 3A-3F in accordance with embodiments ofthe present technology. As shown in FIG. 5, the port selector 126 caninclude a MAC extractor 155 and a MAC filter 156 operatively coupled toone another. The MAC extractor 155 can be configured to extract orotherwise identify a MAC address (e.g., a destination MAC address)included in a header 160 of a packet. Once identified, the MAC extractor155 can be configured to forward the identified MAC address to the MACfilter 156 for further processing.

The MAC filter 156 can be configured to identify a virtual port ID 157based on the MAC address received from the MAC extractor 155. In theillustrated embodiment, the MAC filter 156 can identify the virtual portID 157 by comparing the received MAC address to records of portassignment 162 contained in the memory 124. In certain embodiments, theport assignment 162 can include a table with entries listing a virtualport ID with a corresponding MAC address, a default virtual port ID, orother suitable information. In other embodiments, the port assignment162 can include an index, a state machine, or other suitable datastructures.

FIG. 6 is a block diagram showing example hardware modules suitable forthe RSS engine 128 of FIGS. 3A-3F in accordance with embodiments of thepresent technology. As shown in FIG. 6, the RSS engine 128 can include aRSS hash calculator 172 and a queue selector 174 operatively coupled toone another. The RSS hash calculator 172 can be configured to calculatea hash value based on a characteristic of communication 168 (FIG. 4) ofthe header 160 and a key 168. The key 168 can include a random number orother suitable number unique to the RSS engine 128. Various hashheuristics can be used for calculating the hash value. Example hashheuristics can include perfect hashing, minimal perfect hashing, hashingvariable length data, or other suitable hashing functions. The RSS hashcalculator 172 can then forward the calculated hash vale to the queueselector 174 for further processing. The queue selector 174 can beconfigured to identify a queue in a virtual port and affinitized corebased on the calculated hash value or a portion thereof. For example,the queue selector 174 can compare two least significant bits of acalculated hash value to those included in an indirection table 169 andidentify a corresponding queue/core ID 176 and 177. In other examples,the queue selector 174 can also use the hash value or a portion thereofas the queue/core ID or identify the queue/core ID in other suitablemanners.

FIG. 7 is a block diagram showing certain computing modules suitable forthe load balancer 130 in FIGS. 3A-3F in accordance with embodiments ofthe disclosed technology. As shown in FIG. 7, the load balancer 130include an input module 180, a calculation module 186, a control module184, and an analysis module 182 interconnected with one another. Theinput module 160 can be configured to receive processor parameters 192by accessing debug information from the processor 132 (FIG. 2) or via adebug port and via other suitable interfaces. The processor parameters192 can include core utilization percentage, core active time, coreexecution task identification, or others suitable parameters. The inputmodule 180 can also be configured to receive user input 194 such as ahigh threshold, a low threshold, or other suitable information from anadministrator, a user, or other suitable entities. The input module 180can then provide the received processor parameters 192 and the userinput 194 to the analysis module 182 for further processing.

The calculation module 186 can include routines configured to performvarious types of calculations to facilitate operation of othercomponents of the load balancer 130. For example, the calculation module186 can include routines for accumulating a total time and a totalnumber of packets a core 133 (FIGS. 3A-3F) is used for executing networkprocessing loads of individual queues 139 (FIGS. 3A-3F). In anotherexample, the calculation module 186 can be configured to calculate anoverall utilization of each cores 133. In other examples, thecalculation module 186 can include linear regression, polynomialregression, interpolation, extrapolation, and/or other suitablesubroutines. In further examples, the calculation module 186 can alsoinclude counters, timers, and/or other suitable routines.

The analysis module 182 can be configured to analyze the variousreceived and/or calculated processor parameters to determine whether autilization level of a core is higher than a high threshold or lowerthan a low threshold. For example, the analysis module 182 can compare autilization percentage of a core to the high threshold and to the lowthreshold. The analysis module 182 can then indicate whether the core islikely overloaded or underutilized according results of analysis. Thecontrol module 184 can be configured to control issuance of modificationcommands 190 according to the analysis results from the analysis module182. In certain embodiments, the control module 184 can be configured toissue the modification command 190 to relocate a queue from an originalcore to another core. In other embodiments, the control module 184 canbe configured to coalesce network processing loads from multiple coresto one or a reduced number of cores. Additional functions of the variouscomponents of the load balancer 130 are described in more detail belowwith reference to FIGS. 8C and 8D.

FIGS. 8A-8D are flow diagrams illustrating various aspects of processesfor managing network traffic in a host in accordance with embodiments ofthe present technology. Even though the processes are described belowwith reference to the computing system 100 of FIGS. 1 and 2, in otherembodiments, the processes can be implemented in computing systems withadditional and/or different components.

As shown in FIG. 8A, the process 200 can include receiving packets at anetwork interface via a computer network at stage 202. The packets caninclude headers such as that shown in FIG. 4. The process 200 can theninclude assigning the received packets to a virtual port of the networkinterface at stage 204. In certain embodiments, the received packets areassigned in accordance with a destination MAC address included in theheader of the packets. In other embodiments, the received packets can beassigned based on a virtual machine address or other suitabledestination identifiers.

The process 200 can then include assigning packets in a virtual port ofthe network interface to a particular queue of the virtual port at stage206. In certain embodiments, the packets are assigned to a particularqueue based on a characteristic of communication of the packets. Thecharacteristic of communication can include, for instance, a source IPaddress, a destination IP address, a source port, a destination port,and/or other suitable TCP parameters. In other embodiments, the packetscan be assigned based on other suitable parameters or characteristics ofthe packets. In accordance with embodiments of the disclosed technology,each of the queues can be affinitized to a core of a main processor in ahost. As such, once the packets are assigned to a queue, a correspondingcore can be identified. The process 200 can then include cooperatingwith the core corresponding to the assigned queue to process and forwardthe packets to the particular destination in the general destination atstage 208. An example operation for such processing is described abovewith reference to FIGS. 3A-3F.

As shown in FIG. 8B, the process 210 can include receiving a request toprocess packets in a queue of a virtual port of a network interface atstage 212. In certain embodiments, the packets in the queue can bedestined to an application executing in a virtual machine hosted on aserver. In other embodiments, the packets in the queue can be destinedto the virtual machine, a hypervisor for the virtual machine, or othersuitable software components on the server. The process 210 can theninclude processing packets in the queue in response to the request atstage 214. In certain embodiments, processing the packets can includeinspecting and retrieving the packets from the queue and executing oneor more of enforcing communications security, performing networkvirtualization, translating network addresses, or maintaining acommunication flow state with a core corresponding to the queue. Inother embodiments, processing the packets can include performing othersuitable functions with the core corresponding to the queue.

The process 210 can then include a decision stage 216 to determinewhether the process 210 is complete. In one embodiment, the process 210is complete when the queue contains no more packets. In otherembodiments, the process 210 is complete when a user terminates theprocess 210 or under other suitable conditions. In response todetermining that the process 210 is complete, the process 210 includesterminating operations at stage 218; otherwise, the process 210 revertsto processing additional packets at stage 214.

As shown in FIG. 8C, the process 220 can include receiving processor orcore operating parameters at stage 222. The process 220 can then includecalculating core utilization for executing network processing loads ofvarious queues at stage 224. The process 220 can then include a decisionstage 226 to determine whether utilization of a core is greater than ahigh threshold. In response to determining that the utilization of thecore is greater than the high threshold, the process 220 can includeselecting a new core for relocating one or more queues from the core atstage 228. The process 220 can then include issuing a modificationcommand to, for example, the network interface 136 in FIGS. 3A-3F tomodify affinitization between queues and cores such that the one or morequeues are “moved” from the core to the new core at stage 232.

In response to determining that the utilization of the core is notgreater than the high threshold, the process 220 can include anotherdecision stage 230 to determine whether the utilization of the core islower than a lower threshold. In response to determining that theutilization of core is lower than the low threshold, the process 220 caninclude selecting a new core at stage 228 to relocate queues currentlyaffinitized with the core. In response to determining that theutilization of core is not lower than the low threshold, the process 220reverts to receiving additional processor operating parameters at stage222.

As shown in FIG. 8D, the process 240 can include receiving amodification command at stage 242. The modification command includesinstructions to modify affinitization between a queue in a virtual portof a network interface with a core of a main processor. The process 240can further include modifying an indirection table to effect themodification included in the modification command at stage 246.

FIG. 9 is a computing device 300 suitable for certain components of thecomputing system 100 in FIG. 1, for example, the host 106 or the clientdevice 103. In a very basic configuration 302, the computing device 300can include one or more processors 304 and a system memory 306. A memorybus 308 can be used for communicating between processor 304 and systemmemory 306. Depending on the desired configuration, the processor 304can be of any type including but not limited to a microprocessor (pP), amicrocontroller (pC), a digital signal processor (DSP), or anycombination thereof. The processor 304 can include one more levels ofcaching, such as a level-one cache 310 and a level-two cache 312, aprocessor core 314, and registers 316. An example processor core 314 caninclude an arithmetic logic unit (ALU), a floating point unit (FPU), adigital signal processing core (DSP Core), or any combination thereof.An example memory controller 318 can also be used with processor 304, orin some implementations memory controller 318 can be an internal part ofprocessor 304.

Depending on the desired configuration, the system memory 306 can be ofany type including but not limited to volatile memory (such as RAM),non-volatile memory (such as ROM, flash memory, etc.) or any combinationthereof. The system memory 306 can include an operating system 320, oneor more applications 322, and program data 324. As shown in FIG. 8, theoperating system 320 can include a hypervisor 140 for managing one ormore virtual machines 144. This described basic configuration 302 isillustrated in FIG. 8 by those components within the inner dashed line.

The computing device 300 can have additional features or functionality,and additional interfaces to facilitate communications between basicconfiguration 302 and any other devices and interfaces. For example, abus/interface controller 330 can be used to facilitate communicationsbetween the basic configuration 302 and one or more data storage devices332 via a storage interface bus 334. The data storage devices 332 can beremovable storage devices 336, non-removable storage devices 338, or acombination thereof. Examples of removable storage and non-removablestorage devices include magnetic disk devices such as flexible diskdrives and hard-disk drives (HDD), optical disk drives such as compactdisk (CD) drives or digital versatile disk (DVD) drives, solid statedrives (SSD), and tape drives to name a few. Example computer storagemedia can include volatile and nonvolatile, removable and non-removablemedia implemented in any method or technology for storage ofinformation, such as computer readable instructions, data structures,program modules, or other data. The term “computer readable storagemedia” or “computer readable storage device” excludes propagated signalsand communication media.

The system memory 306, removable storage devices 336, and non-removablestorage devices 338 are examples of computer readable storage media.Computer readable storage media include, but not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other media which can be used to store the desired informationand which can be accessed by computing device 300. Any such computerreadable storage media can be a part of computing device 300. The term“computer readable storage medium” excludes propagated signals andcommunication media.

The computing device 300 can also include an interface bus 340 forfacilitating communication from various interface devices (e.g., outputdevices 342, peripheral interfaces 344, and communication devices 346)to the basic configuration 302 via bus/interface controller 330. Exampleoutput devices 342 include a graphics processing unit 348 and an audioprocessing unit 350, which can be configured to communicate to variousexternal devices such as a display or speakers via one or more A/V ports352. Example peripheral interfaces 344 include a serial interfacecontroller 354 or a parallel interface controller 356, which can beconfigured to communicate with external devices such as input devices(e.g., keyboard, mouse, pen, voice input device, touch input device,etc.) or other peripheral devices (e.g., printer, scanner, etc.) via oneor more I/O ports 358. An example communication device 346 includes anetwork controller 360, which can be arranged to facilitatecommunications with one or more other computing devices 362 over anetwork communication link via one or more communication ports 364.

The network communication link can be one example of a communicationmedia. Communication media can typically be embodied by computerreadable instructions, data structures, program modules, or other datain a modulated data signal, such as a carrier wave or other transportmechanism, and can include any information delivery media. A “modulateddata signal” can be a signal that has one or more of its characteristicsset or changed in such a manner as to encode information in the signal.By way of example, and not limitation, communication media can includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, radio frequency (RF), microwave,infrared (IR) and other wireless media. The term computer readable mediaas used herein can include both storage media and communication media.

The computing device 300 can be implemented as a portion of a small-formfactor portable (or mobile) electronic device such as a cell phone, apersonal data assistant (PDA), a personal media player device, awireless web-watch device, a personal headset device, an applicationspecific device, or a hybrid device that include any of the abovefunctions. The computing device 300 can also be implemented as apersonal computer including both laptop computer and non-laptop computerconfigurations.

From the foregoing, it will be appreciated that specific embodiments ofthe disclosure have been described herein for purposes of illustration,but that various modifications may be made without deviating from thedisclosure. In addition, many of the elements of one embodiment may becombined with other embodiments in addition to or in lieu of theelements of the other embodiments. Accordingly, the technology is notlimited except as by the appended claims.

1-20. (canceled)
 21. A method for network traffic management in acomputing device having a network interface controller operativelycoupled to a main processor with multiple cores, the method comprising:upon receiving, at the network interface controller, packets associatedwith a communication flow, assigning, at the network interfacecontroller, the received packets (i) to a virtual port of the networkinterface controller having multiple queues and (ii) to one of themultiple queues in the virtual port based on parameters of thecommunication flow, the one of the multiple queues being affinitizedwith a first core of the main processor; and while the first core isexecuting network processing operations related to the packets in theone of the multiple queues of the virtual port, receiving, at thenetwork interface controller, a command, from the main processor, toaffinitize the one of the multiple queues with a second core of the mainprocessor instead of the first core; and in response to the receivedcommand, modifying an entry in a table on the network interfacecontroller to indicate that the one of the multiple queues isaffinitized with the second core instead of the first core, therebyallowing the second core to execute additional network processingoperations related to a remaining portion of the packets still in theone of the multiple queues of the virtual port.
 22. The method of claim21 wherein the virtual port of the network interface controllercorresponds to a virtual machine or a hypervisor executing on thecomputing device.
 23. The method of claim 21 wherein assigning thepackets to the one of the multiple queues includes: calculating, at thenetwork interface controller, a hash value of one or more of a source IPaddress, a destination IP address, a source port, or a destination portassociated with the packets; and identifying the one of the multiplequeues as one corresponding to at least a portion of the calculated hashvalue in the table contained in the network interface controller. 24.The method of claim 21 wherein assigning the packets to the virtual portincludes: extracting a destination MAC address from a header of thepackets, the destination MAC address identifies a virtual machineexecuting on the computing device; and identifying the virtual portcorresponding to the extracted MAC address and to the virtual machine.25. The method of claim 21 wherein: the virtual port corresponds to avirtual machine executing on the computing device; and the one of themultiple ports corresponds to an application executing in the virtualmachine.
 26. The method of claim 21 wherein: the virtual portcorresponds to a virtual machine executing on the computing device; theone of the multiple ports corresponds to an application executing in thevirtual machine; and allowing the second core includes allowing thesecond core to executing instructions to enforce communicationssecurity, perform network virtualization, translate network addresses,or maintain a communication flow state to forward the remaining portionof the packets in the one of the multiple queues to the applicationexecuting in the virtual machine.
 27. The method of claim 21 wherein:the packets are a first set of packets having a first header containinga first MAC address; and the method further includes: receiving, at thenetwork interface controller, a second set of packets having a secondheader containing a second MAC address that is the same as the first MACaddress; and assigning, at the network interface controller, thereceived second set of packets to the same virtual port of the networkinterface controller based on that the second MAC address is the same asthe first MAC address.
 28. The method of claim 21 wherein: the packetsare a first set of packets having a first header containing a first MACaddress; the virtual port is a first virtual port; and the methodfurther includes: receiving, at the network interface controller, asecond set of packets having a second header containing a second MACaddress that is different than the first MAC address; and assigning, atthe network interface controller, the received second set of packets toa second virtual port of the network interface controller, the secondMAC address being different than the first MAC address.
 29. The methodof claim 21 wherein: the packets are a first set of packets having afirst header containing a first MAC address and a first destination IPaddress; the one of the multiple queues is a first queue; and the methodfurther includes: receiving, at the network interface controller, asecond set of packets having a second header containing a second MACaddress that is the same as the first MAC address but a seconddestination IP address different than the first destination IP address;assigning, at the network interface controller, the received second setof packets to the same virtual port of the network interface controllerbased on that the second MAC address is the same as the first MACaddress; and assigning, at the network interface controller, the secondset of packets to a second queue in the same virtual port, the secondqueue being different than the first queue.
 30. A computing device,comprising: a main processor having multiple cores configured to executeinstructions independently; a network interface controller operativelycoupled to the main processor, the network interface controller havingone or more virtual port each having one or more queues configured toreceive and temporarily store packets, wherein each of the queues isaffinitized with a core of the main processor; and a memory containinginstructions executable by the main processor to cause the mainprocessor to: receive, at the main processor, a request from the networkinterface controller to perform network processing operations forpackets in a queue of a virtual port of the network interfacecontroller, the packets belonging to a single communication flow; inresponse to receiving the request, perform the network processingoperations at a first core to effect processing and transmission of thepackets; while performing the network processing operations with thefirst core, determine whether the first core has a utilization levelhigher than a threshold; and in response to a determination that theutilization level of the first core is higher than the threshold, switchprocessing and transmission of a remaining portion of the packets fromthe first core to a second core of the main processor.
 31. The computingdevice of claim 30 wherein the request includes a hardware interruptfrom the network interface controller to the main processor.
 32. Thecomputing device of claim 30 wherein: the request includes a hardwareinterrupt from the network interface controller to the main processor;and to perform the network processing operations includes to schedule aremote procedure call on the first core and to cause the first core toexecute the scheduled remote procedure call to enforce communicationssecurity, to perform network virtualization, to translate networkaddresses, or to maintain a communication flow state.
 33. The computingdevice of claim 30 wherein the instructions are executable by the mainprocessor to cause the main processor to: receive, at the mainprocessor, another request from the network interface controller toperform additional network processing operations for the remainingportion of the packets in the queue of the virtual port of the networkinterface controller; and in response to receiving the another request,perform the network processing operations at the second core to effectprocessing and transmission of the remaining portion of the packets tothe application executing in the virtual machine hosted on the computingdevice.
 34. The computing device of claim 30 wherein: the threshold is afirst threshold; and the instructions are executable by the mainprocessor to cause the main processor to: in response to anotherdetermination that the utilization level of the first core is not higherthan the first threshold, determine whether the utilization level of thefirst core is lower than a second threshold when performing the networkprocessing operations to effect processing and transmission of thepackets, the second threshold being lower than the first threshold; andin response to another determination that the utilization level of thefirst core is lower than the second threshold, switch processing andtransmission of the remaining portion of the packets to a third corehaving a utilization level higher than the second threshold but lowerthan the first threshold.
 35. The computing device of claim 30 wherein:the threshold is a first threshold; and the instructions are executableby the main processor to cause the main processor to: in response toanother determination that the utilization level of the first core ishigher than the threshold, determine whether the utilization level ofthe first core is lower than a second threshold when performing thenetwork processing operations to effect processing and transmission ofthe packets, the second threshold being lower than the first threshold;and in response to another determination that the utilization level ofthe first core is lower than the second threshold, switch processing andtransmission of a remaining portion of the packets from the first coreto a third core; and perform additional network processing operations atthe third core to effect processing and transmission of the remainingportion of the packets to the application executing in the virtualmachine hosted on the computing device.
 36. A method for network trafficmanagement in a computing device having a network interface controlleroperatively coupled to a main processor with multiple cores, the networkinterface controller having one or more virtual port individually havingone or more queues configured to receive and temporarily store packets,wherein the method comprising: receiving, at the main processor, arequest from the network interface controller to perform networkprocessing operations at a first core of the main processor for packetsassigned by the network interface controller to a queue of a virtualport of the network interface controller; in response to receiving therequest, perform the network processing operations with the first coreto effect processing and transmission of the packets; determiningwhether the first core has a utilization level lower than a thresholdwhen performing the network processing operations to effect processingand transmission of the packets; and in response to a determination thatthe first core has a utilization level lower than the threshold, switchprocessing and transmission of a remaining portion of the packets in thequeue from the first core to a second core of the main processor. 37.The method of claim 36, further comprising: in response to thedetermination that the utilization level of the first core is lower thanthe threshold, selecting the second core for performing additionalnetwork processing operations for the remaining portion of the packetsin the queue based on a current utilization level of the second corebeing higher than the threshold but lower than another threshold. 38.The method of claim 36 wherein: receiving, at the main processor,another request from the network interface controller to performadditional network processing operations at the second core of the mainprocessor for another portion of the packets in the queue of the virtualport of the network interface controller; and in response to receivingthe another request, performing the additional network processingoperations at the second core to effect processing and transmission ofthe another portion of the packets to the application executing in thevirtual machine hosted on the computing device.
 39. The method of claim36 wherein: the threshold is a first threshold; and the method furtherincludes: in response to another determination that the utilizationlevel of the first core is higher than the first threshold, determiningwhether the utilization level of the first core is lower than a secondthreshold when performing the network processing operations to effectprocessing and transmission of the packets, the second threshold beinglower than the first threshold; and in response to a furtherdetermination that the utilization level of the first core is not lowerthan the second threshold, switching processing of the remaining portionof the packets from the first core to a third core having a utilizationlevel lower than the second threshold.
 40. The method of claim 16wherein: the threshold is a first threshold; and the method furtherincludes: in response to another determination that the utilizationlevel of the first core is higher than the first threshold, determiningwhether the utilization level of the first core is lower than a secondthreshold when performing the network processing operations to effectprocessing and transmission of the packets, the second threshold beinglower than the first threshold; and in response to a furtherdetermination that the utilization level of the first core is lower thanthe second threshold, continuing processing and transmission of theremaining portion of the packets using the first core.